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Abstract — We examine LDPC codes decoded using linear 
programming (LP). Four contributions to the LP framework are 
presented. First, a new method of tightening the LP relaxation, 
and thus improving the LP decoder, is proposed. Second, we 
present an algorithm which calculates a lower bound on the 
minimum distance of a specific code. This algorithm exhibits 
complexity which scales quadratically with the block length. 
Third, we propose a method to obtain a tight lower bound 
on the fractional distance, also with quadratic complexity, and 
thus less than previously-existing methods. Finally, we show how 
the fundamental LP polytope for generalized LDPC codes and 
nonbinary LDPC codes can be obtained. 

Index Terms — Linear programming decoding, maximum like- 
lihood (ML) decoding, low-density parity-check (LDPC) codes, 
minimum distance, fractional distance. 

I. Introduction 

Within the field of error-correcting codes, the method of 
linear programming (LP) decoding of linear codes has at- 
tracted considerable attention in recent years. Over the past 
several years, this method has stirred a substantial amount of 
research. One reason for this is that the LP decoder has the ML 
certificate property [1|, i.e., that if the LP decoder succeeds 
in producing an integral solution vector, then it produces the 
ML solution. Another attractive property of the LP decoder 
is that one may improve the performance in various ways. 
These include using generic techniques to tighten the LP 
relaxations 10, using integer programming or mixed integer 
linear programming (TJ, fl2|, adding constraints to the Tanner 
graph [3 1 and guessing facets of the polytope [4]. 

However, LP decoding is disadvantaged as compared to it- 
erative decoding algorithms such as belief propagation or min- 
sum decoding because the decoding complexity can be much 
larger. In general the LP decoder entails a polynomial, non- 
linear complexity. Several authors have proposed algorithms 
for low-complexity LP decoding, e.g. 0, @, Q. Vontobel 
and Koetter [5| have proposed an iterative, Gauss-Seidel-type 
algorithm for approximate LP decoding. In [8 1 the approach 
of Q was studied under a newly-proposed scheduling scheme. 
This study showed that the performance of the LP decoder can 
be approached with linear computational complexity. It was 
also shown that the iterative LP decoder can correct a fixed 
fraction of errors with linear computational complexity. These 
results were also extended to generalized LDPC (GLDPC) 
codes. 

In this paper, we present the following main results. First, 
we show that merging check nodes in the graph can tighten the 
relaxation and improve the LP decoder. Second, we demon- 
strate how the linear-complexity LP decoder from [8] can be 



used to obtain a lower bound on the minimum distance of a 
specific code, which is also an upper bound on the fractional 
distance of the code, with complexity 0(N 2 ), where N is 
the block length. This lower bound on the minimum distance 
can be improved by using the check node merging technique. 
Third, we propose an algorithm which calculates a tight lower 
bound on the fractional distance with complexity 0(N 2 ). 
We note that Feldman et al. H) were the first to propose 
an algorithm which calculates the exact fractional distance; 
their algorithm entails a complexity which is polynomial 
but not quadratic in the block length. This is also true for 
the fractional distance calculation in (7), even though it has 
reduced complexity as compared to (2. Our approach has 
lower complexity as compared to these techniques. Finally, 
we show how the fundamental polytope of two important 
classes of codes, namely GLDPC and nonbinary codes, can be 
obtained. This is accomplished by using the double description 
method |9), IflOl . In addition to these results, we show how 
the linear-complexity LP decoder can be calculated efficiently 
for GLDPC codes. 

This paper is organized as follows. In section [TT] the linear- 
complexity LP decoder [8] is reviewed. Section [Til] shows 
how to perform some computations relating to the linear- 
complexity LP decoder in an efficient manner. Section [TV] 
describes how improved LP decoding can be achieved by 
merging check nodes. In Section [V] we propose an algorithm 
which yields a lower bound on the minimum distance which 
is also an upper bound on the fractional distance of specific 
LDPC codes. In Section IVI1 an algorithm which produces a 
tight lower bound on the fractional distance is presented. We 
show how to obtain the fundamental polytope of GLDPC and 
nonbinary codes in Section I VIII Section IVIIII provides some 
numerical results, and the paper is concluded in Section [IX] 

II. Approximate Iterative Linear programming 

DECODING OF GLDPC CODES 

Consider a discrete binary-input memoryless channel 
(DMC) described by the probability transition function Q(y\c) 
where c is the transmitted code-bit and y is the channel output. 
Also consider a GLDPC code C represented by a Tanner graph 
with N variable nodes and M linear constraint nodes, where 
each constraint node represents a small local constituent code, 
thus generalizing single parity checks (SPCs) in plain LDPC 
codes. The code C is used to transmit information over the 
given DMC. Following the notation in Qj, let I and J be the 
sets of variable and constraint nodes, respectively, such that 
\I\ = N and \J\ — M, The variable node i G 1 is connected 
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to the set Mi of constraint node neighbors. The constraint node 
j G J is connected to the set Hj of constraint node neighbors. 
Further denote by Cj the vector 



(1) 



Given some j £ J, denote by Cj the constituent binary linear 
code corresponding to the constraint node j G J (thus for 
plain LDPC codes, Cj is a simple parity-check code). Let Oj 
denote the all-zero local codeword on constraint node j, and 
let denote the all-zero codeword in C. A valid codeword 
c G C satisfies Cj G Cj, Vj G J . 

We assume, without loss of generality, that there are no 
parallel edges in the Tanner graph, since otherwise we can 
modify the structure of the code to satisfy this constraint. 

The LP decoder is given by [1], 



We further denote by Q(C) = f] jeJ Qj{Q the relaxed CD or 
fundamental polytope. Thus c is feasible in ©-(O if and only 
if c G 2(C). We thus have, in addition to (0, the relation 



argminP(c) 

cGQ(C) 



(9) 



An important observation is that the LP decoder has the 
ML certificate [ 1 1 in the sense that if the solution of Problem- 
P is integer (in fact an integer u> implies that c is also integer 
by ©) then it is the ML decision. 

Now consider the following problem, which we term 
Problem-D. The variables in this problem are 

and the problem is defined by 



c = argminP(c) 

where the vector uj is defined by 

w = i w ],g}]ej, geCj 



(2) 



and where 



subject to 



P(c) 



W 3,g > 



= c ^ 

iei 



V j G J , g G Cj 

= i Vj e J 



see, 

E 



V j G J , i G Afj 



(3) 

(4) 
(5) 

(6) 



and where 



7i = lo S 



Q(yi I Q) 
Q(yi 1 1) 



where all logarithms henceforth are to the base e. We assume 
that there are no perfect measurements from the channel, i.e. 



OO < 7,; < oo Vi G X . 



(7) 



Note that if we transmit over the binary erasure channel (BEC) 
then this assumption does not hold. However, we can re- 
normalize the 7,'s by replacing ji = oo with ji = 1 and 
7,: = — oo with ji = — 1. After this re-normalization the 
assumption (0 holds, and we can then apply the decoding 
algorithm. 

We term the minimization problem (O-©, Problem-P. We 
also define 

^3 = {Wj,g}gec } 

The polytope Qj(C) is defined as follows 0~): c G Qj(C) 
(sometimes we will use the notation Cj G Qj(C), the meaning 
will be clear from the context) if and only if there exists a 
vector ujj such that dU-© hold for this value of j. For plain 
LDPC codes it was shown that c G Qj(C) if and only if 

< a < 1 Vi G Afj 

E ie Ar,\ S + E, e s (1 ~ *) > 1 V5 C Mj , \S\ odd 

(8) 



maxD(u) where D(u) 



Emin 



u ^ g * 



subject to 



■ 7» 



(10) 



(11) 



Lemma 1: Problem-P and Problem-D are Lagrange dual 
problems. The minimum of Problem-P is equal to the maxi- 
mum of Problem-D. 
The lemma is proven in [8|. 

Vontobel and Koetter [5 | have proposed an algorithm which 
yields an approximated solution to Problem-D. They suggested 
to consider the following "soft" version of Problem-D, termed 
Problem-DS: 

maxDS(u) 



where 

DS(u)4-l^log^ e - K ^ u ^ ,12, 

jej gee, 

subject to (fTTT i. Note that the objective function in ( fl2b is 
obtained from ( TTOb by approximating the min() function 
using a log-sum-exp function with a parameter K > 0. This 
approximation can become arbitrarily accurate by increasing 
K. Also note that DS(u) is concave since the log-sum-exp 
function is convex. In what follows, we give a brief overview 
of the linear-complexity iterative LP decoder. The reader is 
referred to [8| for additional details. 

Let denote the (|/\/}| + 1) x \Cj\ matrix 



\$/3 



(13) 



where l\c-\ is a row vector of \Cj\ ones and the columns of 
tyi are all the codewords of C, . 



Note that the equation set 
1 



expresses the constraints (fS])-©. We define 
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where A T is the transpose of matrix A, and 



r)= max< |Cj||A/}|max|q 

3 I I,' 



where S?. is the element of S 3 . 

Now let e > be a constant such that e < I/77. We define 



A* = A*(l - r?e) + Vi G X 



(14) 



and 



We will also define the quantities 



E 



gec 3 , 9fc =o 

The following iterative algorithm was proposed in j8] to 
find an approximate solution to Problem-P in linear time 
complexity. 

Algorithm 1: Given a GLDPC code, channel observations 
and some fixed parameters eo G (0, 1/??) and K > do: 

1) Initialization: 

Vi G X, j 6JV, : Uij = 7i/|M 

1 B 

Vi e X, j 6 A/"; : Uij = -— log 

Vi elJeATi : A,,, = — — ^ 

Vi G I : e j = max I A; , — A.; I 

A = {iEl : 6i> e } 

2) Iteration: Pick an arbitrary element k G A and make 
the following updates: 



Vj G A4 : u k ,j = Vk,j + 



Ik 1 



J2 Vk " 



Vi £j\fj\k,j GjVfc : Uij = A _ - . ( 



Vi G TV}, j G A4 : Xi d - 1 + e g KiJ -,,,) 
V 4 G AfjJ e M k : Xi = 



Vi G TV}, j G : ej = max |A$ — Aj| 

Vi G TV} , j G JVfc : If i G .4 and ej < e then 4. = A \ i 
Vi G Mj,j G A4 : If i £ 4 and ej > e then 4 = A U i 

3) Loop Control: If A ^ then repeat step 2. Otherwise 
proceed to step 4. 

4) Produce the final solution and Exit: Vi G X : Compute 
Xi using O, and c% using 



Ci 



1 if Xi > 0.5 
if Xi < 0.5 



Output the primal vector A, the dual vector u and the 
bitwise-estimated integral vector {ci}i 6 x 
Note that each edge in the graph connecting the variable 
node i and the constraint node j is associated with the 



variables u 



Vij and Xij. Each variable node i G I is 



associated with the variables A*, Xi 
following assumptions were made. 

1) 

|7t| < 7max 



and ej. In [8| the 



Vi el 



(16) 



for some positive constant 7 max . Note that practical 
channels with non-perfect measurements, that satisfy (0), 
also satisfy ( fT6b . since their output is quantized to some 
finite set. 

2) As was indicated earlier, we can assume without loss of 
generality that there are no parallel edges in the Tanner 
graph. 

3) We assume that dL > 3 V j G J . where d'j is the dual 
distance of the constituent code corresponding to the 
constraint node j . For plain LDPC codes this assumption 
degenerates to > 3 Vj G J (this simpler version 
appears also in 0). 

4) We assume that all node degrees are bounded by some 
constant (independent of N). 

Under these assumptions it was shown in (8|, for any fixed 
e G (0, 1/77) and K > 0, that Algorithm Q] exits after O(N) 
iterations, and that the vector A is feasible in Problem-P. 
Furthermore, by setting K sufficiently large and eo sufficiently 
small, we can make (P(X)—P*)/N, where P* is the minimum 
value of Problem-P, arbitrarily small. More precisely, 

O Theorem 1]: Consider Algorithm Q] under the four 
assumptions above. For any given S, after O(N) iterations, 
each with a constant number of operations, the algorithm 
yields a vector, A, which is feasible in Problem-P and satisfies 



< P(A) -P* < 6N 



(17) 



By weak duality and by the primal feasibility of A, we have 

D(u)<P*<P(A) (18) 

We will make use of the fact that P* is sandwiched between 
the lower bound D(u) and the upper bound P(A). In fact, in 
the proof of JU Theorem 1], it was shown that 



P(A) - D(u) < SN 
and thus follows the result of the theorem. 



(19) 



III. Efficient computations related to the 

ITERATIVE APPROXIMATE LP DECODER 

In this section, we demonstrate how the computation of 
Ak j and -Bfej, defined in ( fl5l ) can be efficiently implemented 
for GLDPC codes. For LDPC codes an efficient computation 
scheme was presented in [8 |. We also show how to efficiently 
implement the calculation of D(u). 

We follow the approach proposed in ifTTI that uses a trellis 
representation of the code. To simplify the notation we assume 



that Afj = {0,1,. 



1}, and we fix j and omit it from 
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Ak,j, Bk.j, Cj and Uij. In addition we assume that the code 
Cj is represented by the parity check matrix 

H = [h h x . . . h n _i] 

with columns ho, hi, . . . , h„_i, such that the vectors h^ are 
m-dimensional binary vectors representing elements of the 
field GF(2 m ). 

A. Efficient Computation of A k j and B k j 

We need to compute A k and B k , k — 0, 1, . . . , n— 1, defined 

by 

A k = ]T e~ K &"=o^ 9 >-u k ) 

S :H S =0,g k = l 



and 



B k 



E 



(20) 



i'i' 



where g = [g , g u . . . , g n _{\ 

Let H k , gfc, E(k,s) and E(k,s,b) be defined by 

H k = [h hi ... h fe ] , g fc = [50,51, •■•,5fc] T (21) 

E(k,s)= J2 e~ K ^=o UiSi 

and 

E(k,s,b) = e- K ^=o^9, 

Sk ■ H k g k =s,g k =b 

= E(k-l,s®b-h k )e- Kuk - b (22) 

In these definitions s is an m dimensional binary vector, rep- 
resenting an element in GF(2 m ), b 6 {0, 1}, g t £ {0, 1}, and 
© is the bitwise exclusive or operator (addition in GF(2 m )). 
Similarly we define 

H k = [hfe+i hfe+2 . . . h„-i] 

gfc = [.9fc+i,5fc+2, ■ • ■ (23) 

E(k,s)= e~ K ^=^ u ^ 

g k ■■ H k g k =s 

E(k,s) and E(k,s) can be computed recursively using the 
forward recursion for k = 0,1, . . . ,n — 1 

Ku k 



B k = 22E(k,s,0)E(k,s) 



(29) 



for k — {),... ,n — 1. 

In practice, in order to avoid numerical problems, the 
forward recursion d24l i and the backward recursion (|25T > are 
computed using logarithmic computation. 

Another possibility is to use the method proposed by 
Hartmann and Rudolph lfl2l . However, in this case one faces 
a numerical problem since the method involves subtractions, 
and not only summations. 

B. Efficient calculation o/D(u) 

To calculate D(u) for GLDPC codes, we use the same 
methodology as in the calculation of A k j and B k j, and we 
use the definitions of H k and in (l2lT i. The difference is that 
here we only need a forward recursion for the calculation. 

By defining 

n-l 

A(k, s) = min > Uigi 
Sk- H k g k =s t-^ 1 
i—0 

it is easily seen that our objective (the minimization in (flOl i) 
is equal to 

A(n- 1,0) (30) 

Now, A(k, s) can be calculated recursively for k = 
0, 1, . . . , n — 1 as 

A(k, s) = mm(A(k - 1, s), A(k - 1, s © h k ) + u k ) 

with the initial conditions 

0, s=0 
oo, otherwise 



A(-l,s) 



(31) 



IV. A NEW METHOD FOR IMPROVED LP DECODING 

Consider some GLDPC code C with an underlying Tanner 
graph Q. Let c be the LP decoded word considered in 
Section HU i.e. 

c = argminP(c) 

ceQ(C) 

where 



E(k,s)=E(k-l,s)+E(k-l,s®h k )-e- KUk (24) Q(C) = 



(for a given value of k we use this recursion to compute 
E(k, s) for all possible states s) and the backward recursion 
for k = n-2,..., 1,0 

E(k, s) = E(k + 1, s) + E(k + 1, s © h fe+ i) • e - Kuk+1 (25) 

and the initial conditions 

1, s=0 



and 



E(-l,s) = 
E(n - l,s) = 



0, otherwise 
1, s=0 



0, otherwise 
Finally, A k and B k are computed using 

A k = e Ktl *^i;(fc,s,l)£(fc,s) 



(26) 



(27) 



(28) 



jc : 3u) = {wj,s} jeJ see, sX - ©' © and © hold } 

Now suppose that we merge some of the constraint nodes 
together to form an alternative representation of the given 
code, with underlying graph Q' . This is illustrated in Figure Q] 
where we merge the constraint nodes j\ and j'2 into j[ 
(i.e. the constituent code associated with j[ satisfies both 
the constraints implied by ji and those implied by j-i), and 
the constraint nodes j$ and j'4 into j' 2 - In this example the 
constraint node j'5 is left unchanged (jg in Q'). 

Denote by J' the new set of constraint nodes corresponding 
to Q' . Now, we can apply the LP relaxation to the new 
representation and calculate c' defined by, 

c' = argminP(c) 

ceQ'(C) 
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Fig. 1 . Starting from the graph Q on the left part of the figure that represents 
the code C, we merge constraint nodes, thus creating a new representation of 
C with an underlying graph Q' shown on the right. 

where 

Q'(C) = |c : 3u; = {w' j J. eJ , tgeC , s.t. ©, © 

and © (with J',C'j instead of J,Cj) hold} 

Proposition 1: Q'(C) C Q(C). 
Proof: Suppose that c e Q'(C). We need to show that 
c G Q(C). First suppose that the only difference between 
Q and Q 1 is that the constraint nodes j\ , j 2 , • ■ • , jr £ J 
in C/ are merged to form the constraint node j' G J - ' in 
C/'. All the other constraint nodes in Q and £?' are identical 
(i.e. they are associated with the same constituent codes and 
are connected to the same variable nodes). Now, there exists 
i w 'i b) c tv «-/./ such that ©' © and © ( with C i instead 
of J, Cj) hold. For j = ji,j 2 , jr, define w jjS by, 

s'eC' pS ' M =s 

where gy-. is the sub-codeword g' restricted to indices i G hfj. 
For all other values of j G J we define u>j. g = w'-,^ g where 
G i s the constraint node in 5' that corresponds to the 
constraint node j G ^7 in 

Note that g' G only if g^ G (since any codeword 
associated with the merged node j 1 is a codeword of Cj for 
all i =ii, . . . , j r ). It follows that for all j = j u . . .,j r , 

w jlg > 
gec 3 g'eq 



gec 3 -, g< =i g'ec< l9 <=i 

Hence we conclude that c G Q(C). We have therefore shown 
that in this case Q'(C) C Q(C). 

Now suppose that Q' is formed by a general merge up of 
constraint nodes in Q. In this case the process of constructing 
Q 1 from Q can be separated to a series of merge up steps of 
constraint nodes, such that at each step we merge a single set 
of parity check nodes from the graph constructed so far in 
order to form a new graph. That is, starting from Q = Q%, we 
first construct Q2 with the polytope Q2. Then we construct Q3 
with the polytope Q3 and so forth, until we obtain Q s = Q 1 
with the polytope Q s = Q'(C). For i = 1, 2, . . . , s - 1, the 
graph Gi+i is obtained from Gi by merging a single set of 
constraint nodes into one constraint node and leaving all the 
other constraint nodes intact (e.g., in the example shown in 
Figure [T] Qi = Q is the graph shown on the left. We first 
merge ji and j 2 into j[, thus forming Q 2 - Then we merge 
j3 and j'4 into j' 2 thus forming Q3 = Q', which is the graph 
shown on the right). By what we have shown so far we know 
that, 

Q'(C) = Q s C Q s _! C...CQ 1= Q(C) 

□ 

Note that when we merge all the constraint nodes into one 
constraint node, the resulting polytope, Q ML (C), is the convex 
combination of all possible codewords. That is, the set of 
vertices of this polytope is the set of codewords of C, and 
the minimization of P(c) over c G Q ML (C) yields the ML 
decoded codeword. Thus we have Q ML (C) C Q'(C) C Q(C). 
Note also that for any codeword c we have c G Q(C), 
c G Q'(C) and c G Q ML (C). That is, Q'(C) is a finer 
relaxation of Q ML (C) than Q(C). 

Let the error probability P e (c) of the decoder that uses the 
initial graph Q, given that the transmitted codeword is c, be 
defined by, 

P e (c) = Pr(3c e Q(C) : c ^ c, P(c) < P(c)) 

Similarly, the error probability P e (c) of the decoder that uses 
the graph Q', given that the transmitted codeword is c, is 
defined by, 

P e '(c) = Pr(3c G Q'(C) : c ^ c, P(c) < P(c)) 

By proposition [TJ we immediately obtain the following, 

Proposition 2: P' e (c) < P e (c). 
This proposition holds trivially due to the fact that the second 
LP decoder uses a tighter relaxation of <2 ML (C). 

Proposition 3: Suppose that in the merging process of 
constraint nodes we only merge constraint nodes ji, . . . , j r 
such that the subgraph of Q induced by ji, . . . , j r and their 
direct neighbors is cycle free. Then Q(C) = Q'(C) and 
P e (c) = Pi(c). 

Proof: We already know from Proposition Q] that Q'(C) C 
Q(C). Thus we only need to show that Q(C) C Q'(C). We first 
prove this assertion for the case where Q 1 is constructed from Q 
by merging only a single pair of constraint nodes, ji , j 2 G J , 
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of Q. In this case, by the assumption of the proposition, there 
are two possibilities. 

1) There is no overlap between the variable nodes that 
are connected to j\ and the variable nodes that are 
connected to ji- Denote by Ci and C2 the sub-codewords 
associated with the variable nodes connected to ji and 
ji respectively. In this case there is no overlap between 
Ci and C2. This case is illustrated in Figure [2] 

2) Exactly one variable node is connected to both ji and 
]2, that is there is an overlap of one bit, C1.2, between 
ci and C2. This case is illustrated in Figure [3] 



0} for g G Cj such that 




merge and create f 



Fig. 2. Merging two parity nodes, j\ and ji, in the Tanner graph of the 
code C. In this case there is no overlap between ci and ci. 




n 



J2 



merge and create j' 



Fig. 3. Merging two parity nodes, j\ and ji, in the Tanner graph of the 
code C. In this case there is an overlap of one bit, c\,i, between ci and ci. 

Now consider the first case (Figure |2). Suppose that c G 
Q(C). Then for j — there exists a set of weights {wj tS > 



E w ^s = 1 



and 



u Jig Vie Afj 



gSCj , gt =i 

Define the set {wf ig } for g = (gi,g2), where gi and g2 
correspond to the sub-codewords Ci and C2 respectively in 
Figure 12 by 

W j',(gl:g2) — W jl,Sl ' W j2,g2 

It follows from this definition that Wj' ( gl ,g 2 ) > 0, and 



Wj 



1 



E E E 

Furthermore, for i G Afj 1 

E w J'.s = E w J',(suS2) 
gecj (gi,g 2 )ec; 

Si = l 9i,i=l 

= 12 w ^si 12 

9i,i=l 

where g lti (g2,i, respectively) is the i'th component of gi (g 2 ). 
Similarly, for i G Afj 2 



w 



J2,g2 — L « 



E 



"3 ,g ~~ u » 



This shows that in the case considered c G Q'(C). 

Next consider the second case (Figure [3]). Suppose that c G 
Q(C). Then for j = ji,j 2 there exists a set of weights {wj tS > 
0} for g G Cj such that 

12 w J,s = 1 C 32 ) 
geCj 

and 

Q = w, :g V z G A/} (33) 

Let 61 denote the sub-codeword Ci excluding the common 
bit ci,2- Similarly 62 is the sub-codeword C2 excluding ci,2- 
Define the set {wj', s } for g = (§1,51,2, £2), where gi, g 1<2 
and g2 correspond to 61, C12 and £2 respectively, by 

A 

w i',s — w i',(gi,si,2.g 2 ) = 

w h,{sui) ■ W h,0-,Sz) M,2 if 51,2 = 1 

w ji,(ii,o) ' w j2,(o,g 2 )/(l - ci, 2 ) if .gi,2 = 
It follows from this definition that Wf jS > 0, and 

12 W 3'S 

see; 

1 



E E 



31,(gl,0)%2,(0,g2 



1 Cl ' 2 si ■ (gi.ojeCjj g 2 : (o,g 2 )ec J2 

~ E E W il,(gl,l) W J2,(l,g2) 

' gi : (gi,l)6C 31 g 2 : (l,g 2 )eC J2 



= 1 
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where the last equality follows from the following equalities, 
which are implied by d32l i and (|33l l, 

c i,2 = E w iu(siA) = E u, J2,(i,g 2 ) 
gi:(gi4)ec 31 g 2 : (i,g 2 )ec j2 

(34) 

1-Cl )2 = W Ji.(gi,0)= E W j2,(0,g 2 ) 

gi : (gi.ojeCj! g 2 : (o,g 2 )ec 32 

(35) 



It remains to show that 



E 



Wj 



(36) 



gee' , 9i =i 



This is shown by considering the following three possibilities 

for i: i £ N n n A/j 2 , i £ N n \ N n and i £ A/} 2 \Afji. In the 
first case, i £ Afj 1 D A/} 2 , we have cj = ci^. In this case 



E 



gec^ (gi,i,# 2 )ecj 

9i=l 



g 2 ) 



— E E 



a node to merge with node j. If there exists some node j' £ V 
such that A/j fl A/j/ 7^ then j' is selected to be merged 
with j. Otherwise, j' £ V is picked arbitrarily. Now j and j' 
are merged into a new constraint node After performing 
the merging operation on these two nodes, a new graph Q( 2 ' 
is formed with the corresponding polytope, Q^ 2 \C). Now, 
in Q, the two constraint nodes selected for merging either 
have no common variable node or are connected to a single 
common variable node. By what we have already shown, we 
know that Q^'(C) = Q( 2 \C). Now we repeat the process. 
If there is a node j" within the remaining nodes in V such 
that A/j(i) fl A/j// 7^ then j" is selected to be merged with 
jW, otherwise j" is selected arbitrarily. Once j" is chosen, it 
is merged with j'W into a new constraint node p 2 ', forming 
a new graph with its corresponding polytope, Q^\C). 
By this selection process, in G^ 2 ', the two constraint nodes 
selected for merging again have either no common variable 
node or are connected to a single common variable node, and 
thus Q^(C) — Q^(C). This merge up process continues 
until we have created the graph Q' = with polytope 

Q'(C) = QW(C). Thus we have shown that, 



Cl,2 



C1.2 



gl : (81,1)60,1 



g 2 : (i,g 2 )ec 32 



where the last equality follows from ( |34| i. 
In the second case, i £ JVj 1 \ Nj 2 , we have 



E w i',s = E 
gecj (gi,i,g 2 )ec; 

9»=1 91,1=1 



E 



Cl,2 



U Jl,(gl,0) ' ^2,(0,62) 



(gi:0,g 2 )ec; 

9l,i = l 



E 



1 - Ci ■ 



y jl:(gl,l) 



Cl,2 



E 



w j 2 ,(i,g 2 ) 



gi : (gi.ijec^ ' g2 : (i,g 2 )ec J2 

E w ji,(gi,o) 
i_ Cl : w j 2 ,(o,g 2 ) 

gi : (ii,o)eC;,i ' g 2 : (o, g2 )ec J2 

9l,i=l 

E ^1,(41,1)+ E w ii,(gi.o) 

gi : (gi,l)GC n gi:(gi,0)eCji 
91,»=1 9i,*=l 



where the third equality is due to ( f34b and d35l l. and the fourth 
equality is due to d33l l (with j = ji). The proof that ( f36l > holds 
in the third case, i £ Nj 2 \ Afj 1 , is identical. Thus in all three 
cases considered d36l l holds, and we conclude that c £ Q'(C) 
as claimed. 

We thus conclude that when we merge a single pair of 
constraint nodes, Q(C) = Q'{C). We proceed to prove that this 
holds for a general merge up of a set of constraint nodes V = 
{ii i • • • j ir}> for which the subgraph of Q induced by the nodes 
in V and their direct neighbors is cycle free, into one node j'. 
We denote the new graph by Q' . We start from the original 
graph Q with the corresponding polytope Q(C) = Q^(C), 
and select a constraint node j £ V arbitrarily. Now we select 



Q(C) = Q (1) (C) = QS Z} {C) 



(2), 



Q (r, (C) = 0!{C) 



The proof of the general case, where we merge several 
non-overlapping groups of variable nodes in Q in order to 
form Q 1 (e.g., in the example shown in Figure [T] where we 
merge j% and ji into j[, and j'3 and j'4 into j^), now follows 
immediately. Thus we have shown that Q(C) = Q'(C). This 
implies that P e (c) = P' e (c). □ 

Proposition [3 implies the following. Consider some graphi- 
cal representation Q of a code C and suppose that we wish to 
obtain a new graphical representation Q' by merging constraint 
nodes such that when using LP decoding, the error probability 
of Q 1 is smaller than the error probability of Q. Then, when 
considering candidate constraint nodes for merge up, it is 
sufficient to consider only constraint nodes j% , . . . , j r such 
that the subgraph of Q induced by jx, . . . , j r and their direct 
neighbors contains a cycle. 

In the sequel, we will use a variant of Proposition [3] where 
upon merging nodes ji, j2, ■ . ■ , j r into j', we keep the original 
nodes j% , j'2, . . . , j r as well as the new node j' (the reason for 
this is that we want to enable some of the nodes jx, j'2, . . . , j r 
to be merged with other nodes). Due to Propositions Q] and [3] 
keeping ji, J2, ■ ■ ■ ,jr has no effect on the resulting Q'(C) and 

V. Bound on the Minimum and Fractional 
Distance of Specific Codes 

It was already shown in [l] that LP decoding can be used to 
obtain the fractional distance, which is also a lower bound on 
the minimum distance of specific linear codes, in polynomial 
time complexity. In this section, we show how Algorithm [T] 
can be used to obtain a lower bound on the minimum distance 
of a given code, and that this bound is also an upper bound 
on the fractional distance. Assuming the degree of the check 
nodes is bounded by a constant independent of N, this 
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procedure requires execution of Algorithm[T]a number of times 
proportional to N, and thus the total complexity is 0(N 2 ). 

Let r G J be a check node and let c r be some nonzero local 
codeword of r. The idea is to search for a minimum-weight 
vector subject to the constraint that c r is the local codeword on 
node r (later we will show how such vectors allow to obtain 
a lower bound on the minimum distance). This conforms to 
the setting of Problem-P, if we set 74 = 1 for all i. Denote 
I r = I\M r and J r = J\\r\. We would like to obtain the 
solution by running Algorithm [T] on a modified version of 
the graph in which we remove the node r, all its neighbors 
M r and the edges connected to these neighbors. However, in 
order to maintain a correct representation of the code, we must 
account for the specific nonzero local codeword c r . To do this, 
we look at all constraint nodes j £ J r where Mj D M r ^ 0. 
Let j € J r be such a constraint node described by a matrix 
Hj with columns {hjj-jgjv' . ; this situation is exemplified in 
Figure [4(a)] Since we will be forcing the variables in Mj f]M r 
to a value depending on c r , we have that the local codeword 
vector Cj G Cj on node j satisfies 



E - 



E 



(37) 



We now remove parity check r and all its neighbors from 
the graph. Consequently, in the remaining graph we observe 
that check node j describes a coset code where a stan- 
dard constraint of the form HjCj = is replaced by the 
= h r j Cr , where HJ = {hj ^-y y; and 
{ c i}i£Nj\N r - This is exemplified in Figure [4(b)| Denote 
the set of local codewords of this coset code by Cj' Cr . Also 
denote c( r ) = {ci}i e z r , o/ r ) — {^j, s }jej T .. s ec r - Cr - Problem- 
P r,Cr is defined as follows. 



constraint ifjcj 



min y 1 

cW,wW 

z<ax r 



subject to 



w 



J-S 



> Vj ej r ,ge C 



E 



= 1 



V j e Jr 



(38) 

(39) 
(40) 

(41) 



Cj = Wj iS Vj € J r , i £ Mj \ Mr 

sec^ , 9i =i 

Thus, by setting ji = 1 Vi S I, Problem-P r ' Cr is easily 
seen to be an instance of Problem-P, and thus its solution can 
be approximated arbitrarily closely by Algorithm CJ in linear 
time. The only modification we need to make is to account 
for the coset codes {Cj' Cr }, and this can be accomplished by 
replacing the definition of Ak,j and Bkj in (15[ by 



e -^E i6 ^\(Ar r u{s ! })"i,iS< 
gec; c '', 3fc =o 

These can be computed efficiently for any coset code Cj' Cr 
using the method described in Section ITlI-AI However, instead 




H, 



' 1 








T 







1 


1 


1 




1 


1 











(a) 




" 1 





" 




" 1 " 





1 


1 


s r = 


1 


1 


1 











(b) 



Fig. 4. Coset codes formed when a codeword c r = (0, 1, 1, 0) is forced 
on constraint node r which is removed, (a) The rightmost column of Hj 
corresponds to a removed variable node forced to a '1' value, (b) The new 
code Cj' Cr is formed as r and Mr are removed from the graph. 



of < f20b we now have (note that for simplicity we omit the 
dependence on j) 

Ak = ]T e -*&o «i«-«0 

g : H r g=h r c r, g k =l 

B k = e - K ^=° Uiai . (43) 

g : ff r g=h' •<='•, g k =0 

It can be seen that the only alteration we need to make is to 
replace ([28j and $29$ by 

(44) 



and 



S 



(45) 



for k — 0, . . . , n — 1. For the calculation of D(u) of a 
coset code, we can appropriately modify and use the efficient 
method from Section IIII-BI The necessary modification for a 
coset code characterized by a vector h r Cr , is that we need to 
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recursively calculate 



A(n- l,h r >' 



(46) 



rather than A(n - 1,0). 

Solving Problem-P r ' Cr thus finds an approximate minimum- 
weight vector c subject to enforcing a local codeword on parity 
check r. We repeatedly apply this method, solving Problem- 
P r ' Cr over all nonzero local codewords c r E C r \{0 r }, and 
for each local codeword we record both a lower bound 
l r c = D(u) and an upper bound = P(A) (see ([T8T l) on the 
optimal value. In this manner, we create two sets of values 

{ l l r }c r ec r \{o r } and {IjT} Cr eC r \{o r }- Next ' we calculate the 
minima 

c_ (47) 



/ m i n = mm 

nun . r _ 

— CreC r \{O r } — 



and 



IL„ = min l r c 
mm c r ec r \{o r } c 



(48) 



Finally, the entire procedure is repeated for all r 6 J and we 
calculate 

kis. = min{^ in } , Z mi „ = min{|^~} (49) 

We refer to this process as Algorithm^ which is summarized 
in what follows. 

Algorithm 2: Given a GLDPC code, do: 

. loop over r e J 

- loop over c r e C r \{0 r } 

* loop over i e J\f r and j e M 
■ Compute h^' Cr using ( f37l > 

• Define FJ = {hi} ie jv>\jv r 

• Compute all codewords of the local coset code 
Cj' Cr defined by HjC r — h^' Cr 

* Run Algorithm [T] to solve Problem-P r c '', defined 
by (f38T>-(f4TT>. While running the algorithm, use the 
definitions of Ak,j and Bkj in (l42l instead of 
those in ( fT3b 

* Set Z£ = D(u) and = P(A) where the vectors 

u and A are output by Algorithm Q] Calculate 
D(u) using ([Tol l or by the more efficient method 
in Section IIII-BI with (|46j instead of (l30t . 



- Calculate F 



,ec r \{o r } 



Z and ZL 



i c ,.ec r \{o r } l c : 



. Output Zjnin = mm reJ {l^ in } and Z min = rriuvg^ZJ^J 
Proposition 4: l m - m , output by Algorithm [2] is a lower 
bound on the minimum distance of the code. 

Proof: First, we claim that l^ in , defined in d47i i. is a lower 
bound on the minimum distance of the code, subject to the 
restriction that the local codeword on node r is nonzero. This 
is due to the fact that a minimum-distance codeword c r subject 
to this restriction must have some nonzero local codeword 
c r on check node r, and while running Algorithm [T] with 
the values of c r forced on variable nodes Af r , we optimize 
over a set containing c r . Furthermore, due to weak duality, 
the result l r c = D(u) is a lower bound on the optimal 
value. Consequently, l r mia is a lower bound on the minimum 
distance subject to the above restriction. Now, since a (global) 



minimum-distance codeword c must contain a check node r', 
the neighbors of which represent a nonzero local codeword <v, 
then at some point Algorithm [2] will have passed through node 
r' and the local codeword c r > and produced a lower bound on 
the weight of c. Since we take the minimum value in d47b and 
d49l ), l m i n is a lower bound on the minimum distance of the 
code. ■ 
Recall that we assume parity check degrees which do not 
depend on N. Consequently, for any check node r, the bounds 
l^ in and Z^ in are obtained by running Algorithm Q] a constant 
number of times. Repeating this process over all check nodes 
thus entails a total complexity of 0(N 2 ). 

Next, we claim that l mm is at least as large as the fractional 
distance. 

Proposition 5: The bound l m - ln output by Algorithm|2]is an 
upper bound on the fractional distance of the code. 

Proof: Recall that Z m i n was obtained by running Algo- 
rithm Q] over some reduced version of the code, corresponding 
to the selection of a nonzero local codeword c r on some 
check node r. That is, we find the minimum of Y^iex Ci 
over the original LP polytope Q(C) while also imposing the 
values c r of the local codeword. For c r = {c r i}ieM-> ^ 
Q Cr {C) = {(pi,...,p N ) G Q{C) : Vz e Pi = c r>i } 
be the polytope Q(C) with the added restriction of the local 
codeword c r . Now, the minimum of any LP is attained at a 
vertex. Consequently, if we show that V(Q Cr (C)) C V(Q(C)), 
where V("P) is the vertex set of the polytope V, then the value 
of imin output by Algorithm [2] is an upper bound on the L\- 
norm of a nonzero pseudocodeword in C, and is thus an upper 
bound on the fractional distance. To this end, it will suffice 
to show that imposing an integer value on a single coordinate 
does not create new vertices. Without loss of generality, let 
Q N (C) 4 Q(C)f]{c N = 1}, and v e V(Q N (C)). We 
now show that v 6 V(Q(C)). In essence, an easy algebraic 
argument will be used. First, note that Q(C) can be represented 
as a matrix inequality 



Ac y b 



(50) 



where c and b are column vectors, A is a matrix representing, 
together with b, the constraints (|8]Q for all j £ J, and >z is 
the standard coordinate-wise inequality. The polytope Q (C) 
can be similarly represented by 



A'c y W 



where 



A' 



A 







1 





' b " 


, b' = 


1 





We make use of the property that a vertex is the intersection of 
N independent hyperplanes. Specifically, if v £ Q N (C) then 
v G V(Q (C)) if and only if there exists a subset of N rows 
A[ of A', and a corresponding sub vector b^ of b' such that 

'The constraints (8), which are valid for plain LDPC codes, are clearly 
expressible in matrix form. In Section \VU\ we will see that for GLDPC codes, 
the fundamental polytope is also expressible in the form {5Q\ , and thus the 
conclusion of Proposition \5\ holds also for GLDPC codes. 
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and det(Ai) ^ (see, e.g., [13] p. 185]). Suppose v e 
V(Q N (C)). If the matrix A[ does not contain the last row 
of A', then it is also a sub-matrix of A and we conclude that 
v G V(Q(C)), as required. If A\ contains the last row of A', 
then we replace this row in A[ with 

[0 ...0 - 1] (51) 

which appears in A due to the constraint cm < 1 (see ([H}); 
with this replacement the determinant is still nonzero and we 
conclude that v G Q(C) is the intersection of N independent 
hyperplanes of Q(C), and thus v G V(Q(C)). ■ 

Discussion. In this section it was shown that Z m ; n and 1^, 
output by Algorithm |2] and obtained with complexity 0(N 2 ), 
constitute a lower bound on the overall minimum distance 
and an upper bound on the fractional distance, respectively. 
Due to $1% and the proof of [8 Theorem 1], it can be seen 
that l m \ n — l m [ n < SN where 6 > can be made as small as 
desired. Moreover, it is possible to improve the lower bound 
on the minimum distance in the following two ways. First, the 
node merging technique presented in Section [TV] can be used 
to produce a tighter LP relaxation, and thus a better lower 
bound on the minimum distance. This approach is exemplified 
in Section lVHIl Second, instead of examining single constraint 
nodes on which a local codeword is forced, one could divide 
J into pairs of constraint nodes, and for each pair examine all 
nonzero combinations of their local codewords. For each such 
nonzero combination of pairs of codewords, it is possible to 
find a minimum-weight vector subject to forcing the variable 
node neighbors to values corresponding to these codewords, 
similar to what is done in Algorithm [2] As another possible 
variant to Algorithm |2j we propose the following greedy 
procedure, which we term the bar method. 

We set a target level or "bar", denoted BAR, for the lower 
bound which the procedure attempts to exceed, as follows. We 
loop over all check nodes, as in Algorithm [2] For each check 
node rejwe evaluate l^ in , defined in (07]). If l r min > BAR, 
we proceed to the next check node. If < BAR, say due 
to a local codeword cj? for which l c a < BAR, we refine the 
bound l c o as follows. First we pick a check node j G Mi 
for some i G M r \ thus j is a check node at distance 2 from 
r, with respect to Q. Next we loop over the local codewords 
{cj} on node j which coincide with c" (i.e., which have the 
same values on the variable nodes in Mj C\M r ). For each such 
local codeword, we run Algorithm Q] forcing local codewords 
on both check nodes r and j. This forcing of local codewords 
jointly on a pair of check nodes produces a lower bound l c o Cj 
on the minimum distance subject to forcing both check nodes 
to these local codewords. Define = min Cje c ] {l c o Cj : 

■ 

Cj coincides with c®}. If > BAR, we conclude that the 
bar has been exceeded for check node r, and proceed to the 
next check node. Otherwise, we pick another check node j G 
Mi for some i G M r and repeat the process until for some 
choice of node j at distance 2 from r the bar is exceeded. If 
the bar is not exceeded for all nodes j at distance 2 from 7-, the 



procedure is terminated with failure. If, on the other hand, for 
all r G J we have l 7 min > BAR or otherwise if for all c° G C r 

c° j 

we have I ■ > BAR for some node j at distance 2 from r, 

111 111 — 

then we say that the level BAR is attainable. Due to the bound 
refining process, an attainable bar level is guaranteed to be a 
lower bound on the minimum distance. Using initial lower and 
upper values for the bar (which can be determined ad-hoc), 
we can run the procedure, each time using a different level 
for the bar, and use the bisection method to find the highest 
attainable bar level. 

VI. A Lower Bound on the Fractional Distance in 
Quadratic Complexity 

In HI, the following algorithm for calculating the fractional 
distance <ifr ac was proposed. Consider the codeword polytope 
Q(C) and the all-zero vertex 0. It was shown in IT) that 
the facets of this polytope (for plain LDPC codes) are given 
by ©, Vj G J. Denote the set of facets of Q(C) which 
do not contain by T. For each facet / G T, run an LP 
solver to find the minimum L\ norm J2iei Ci ovel /• The 
smallest value (over all facets) obtained in this procedure is 
the fractional distance UJ. The complexity of this calculation 
is the same as the complexity of running an LP solver J 7 ! 
times, and in our case we have IJ 7 ) = O(N). By using the 
iterative linear-complexity LP decoder, we will demonstrate 
a procedure which produces a lower bound on the fractional 
distance with complexity 0(N 2 ). In terms of computational 
complexity, this compares favorably with the aforementioned 
procedure because an LP solver in general has complexity 
worse than O(N). Furthermore, if a higher complexity is 
allowed, this lower bound can be made arbitrarily close to the 
true fractional distance. In this section we assume plain LDPC 
codes. The results in the next section enable the generalization 
of the algorithm to GLDPC codes. 

Recall that c G Q(C) if and only if < c, < 1 Vi G 1 and 
Vj G J, S C Mj, \S\ odd we have 

E c,+^(l- Cl )>l 

Of these inequality constraints, the facets of Q(C) which do 
not contain are 

{c, - l} ie i (52) 

and 

c* + £(l-Ci) = l 

Vj eJ.Sc Mj, \S\ odd, |5| > 1 (53) 

We follow the approach in flTJ, i.e., for each of the facets in 
j52l)-(l53l) we evaluate the minimum L\ norm over the facet. 
We show how each such evaluation can be performed in linear 
time using Algorithm [JJ 

Let i G 1 be some index and consider its corresponding 
facet from d52l ). To find the minimum weight over this facet, 
we can implement the method from SectionJVJ as follows. The 
Tanner graph is modified by removing variable node i and all 
the edges incident to it. Now, to keep the graph consistent 
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with the original code, each constraint node j £ Mi (be it 
a standard parity check or generalized node) is modified so 
that it represents a local coset code Hj^Cj^ = h; j (where 
the subscript j\i denotes that we remove the i'th column of 
Hj and the i'th bit of Cj) for some column vector hij ^ 
rather than HjCj = 0. The column hj^ is the column from 
the parity-check matrix representing constraint node j, which 
corresponds to the index of variable node i E Afj (note that 
if j is a standard parity check then j = 1 has dimesion 1). 
Algorithm[T]runs the same on this modified graph, except that, 
similarly to Section [V] we need to replace the calculation of 
Ak,j and B k j in d28j and (|29]l by 04) and (1451 . respectively. 
The vector hjj replaces h r c '- in (|44t and d45) . If we take 
the value of the dual in the execution of Algorithm Q] then 
due to ( [181 1 we get for each i G X, a vector with weight d^ c i 
which is a lower bound on the minimum weight over the facet 
{ci — 1}. The minimum value, 

^=™*<ffi,< (54) 

is thus a lower bound on the minimum fractional weight over 
all facets d52i > of Q(C). This lower bound can be made as tight 
as desired if in Algorithm Q] we take K large enough and eo 
small enough. 

We now turn to the problem of calculating minimum 
fractional distance on the facets (|53T >. Let r be some fixed 
constraint node and let S C Af r be an odd-sized set, \S\ > 1. 
Define the hyperplane R r,s as follows: 

R rS =\c: Y, Ci + X)(l-c*) = l| 
[ ieAf r \s ies J 

Our problem is to find the minimum fractional distance on 
R r ' S fl Q{C), denoted d r { ^ c . Consider the following problem, 
Problem- P^ c ' B : 

mm^c,|5 c * + ~ Cj ) _ 1 (55) 

c ' iei \ieNr\s ies / 

subject to ©-(O, where B > is some large constant. This 
definition is motivated by the following observations. First, 
the feasible region of Problem-Pf r ' a ^' S (in the c variables) is 
by definition the polytope Q(C). Thus, every feasible point 
must satisfy 

Y c 4 + ^(l- Cl )>l (56) 

The second term in ( f55b is, by ( 156] ). a positive penalty term 
on the event of sharp inequality in ( 1561 ). Therefore, in the 
limit where B — > oo, the exact value d^ ac is produced as 
the solution to Problem-Pf ra ^' S . If the constant B is finite, 
the weight dl^. s B of the solution to Problem-P^' 5 can be 
seen to be a lower bound on <££l. (since the vector c which 
attains the minimum fractional distance d£ ac is feasible in 
Problem-P t r ra ^' B and yields the objective function value 
Second, the objective function of Problem-P^' S is linear in 
its variables (it also contains the additive constant B(\S\ — 1) 
which is independent of the variables and thus can be ignored). 



Consequently, Problem-Pf r ^' B can be reduced to an instance 
of Problem-P, by setting {7i}igi as 

(l i<£Af r 
% - < l-B ieS (57) 
( 1 + B ie N r /S 

We conclude that the solution to Problem-P^' B can be 
approximated arbitrarily closely by Algorithm Q] Now, we 
use Algorithm □ to solve Problem-P^' 5 Vj e J,S C 
Afj,\S\ odd, | S | > 1. By taking the dual value output 
by Algorithm Q] again by (1181 . we obtain a set of lower 
bounds {dj^ rSB } on the minimum fractional weight for 
each of the facets d53l l of Q(C). A lower bound on the 
minimum fractional distance over the facets (l53l l is given 
by 

combine this result with ( l54l l and obtain the lower bound 

4ac > rffrac.B = min(d^, <^. )B ) 

Assuming the node degrees in the Tanner graph are bounded 
by a constant independent of N, the overall number of calls to 
Algorithm [TJ is O(N). Thus the computational complexity of 
evaluating a lower bound on the minimum fractional distance 
using the procedure described above is 0(N 2 ). 

Naturally, the exact fractional distance can be approached 
by using very large values for the penalty constant B. It 
should be noted that this could theoretically have an effect 
on the bound on the convergence rate of Algorithm [T] In J8] 
Theorem 1] it was shown that the bound on the convergence 
rate of Algorithm Q] is related to 7 max , and from d57l i it can 
be seen that this quantity may be large as we increase B. 
In Section IVIIII we describe several experiments conducted 
with B = ION. In these experiments, we have not observed 
a significant increase in running time as compared with the 
minimum distance bounds from Section [V] 

VII. Fundamental polytopes of GLDPC and 

NONBINARY CODES 

For plain LDPC codes, the fundamental polytope is repre- 
sented by ©. In this section, we propose a practical procedure 
which obtains representations of the fundamental polytopes 
of two important classes of codes: GLDPC and nonbinary 
codes. Using these representations, one can apply a procedure 
similar to the one presented in Section [VI] to calculate a tight 
lower bound on the fractional distance which can be used to 
assess the performance of the LP decoder in these cases. An 
interesting effort for nonbinary codes has recently been made 
by Skachek [14], for trenary codes. We propose a general prac- 
tical technique which relies on the double description method 
[9 1, [ 10] to find, similar to (|8), a description of the fundamental 
polytope. For the case of nonbinary codes, as in [14], we 
consider the LP formulation from lfl5l which expresses a 
nonbinary decoding problem using a binary problem of higher 
dimension. 

Consider a constraint node j. Suppose it represents some 
general, not necessarily linear, local code Cj with M code- 
words and block length d (in what follows, we will omit the 
subscript j). Similar to ( fl~3T > we construct a d x M matrix ^ 
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for this code by writing all the codewords on the columns of 
that matrix, i.e. 



* = ( c° c 1 ... c" 
where c°, c 1 , . . . , c M_1 are the codewords. We also define 

1m 



,M-1 



f = 



where 1m is a row vector of M ones. The code poly tope, 
Q(C), is defined by (see Section [TVl for an equivalent defini- 
tion) 



Q{C) = I c : 3uj y such that = 



1 
c 

Following |[T6l Theorems 4.9 and 4.10, pp. 97-98], we now 
show that Q(C) is indeed a polytope, and present it in a more 
explicit form. Consider the following two systems of linear 
inequalities 

! 1 (58) 



and 



u) >- , ^u) 



* T v < , (1 c T )v > 



(59) 



By Farkas' lemma (e.g. |fT71 page 263]) the systems (|58l 
and ( 1591 are strong alternatives. That is, the system (IB"8l 
is feasible if and only if the system d59l is infeasible. We 
conclude that, 

Q(C) = {c : (1 c T )v < Vv e T} (60) 

where T is the following polyhedral cone 

T = [v : # T v < o| (61) 

Now, T can be expressed in the following alternative form, 



T = I v : v = MiV( , {vi > 0}[ =1 



(62) 



1=1 



where v;, I = l,...,r are the extreme rays of T. By 1 16 
Proposition 4.3, p. 94] v; is an extreme ray of a polyhedral 
cone T if and only if {^v; : r\ > 0} is a one dimensional 
face of T. For the polyhedron T C M d+1 , the vector v 6 7~, 
v 7^ is an extreme ray if and only if it satisfies d linearly 
independent constraints among ty T v ^ with equality. 
By (|60]l and d62]i we have 



Q(C) = {c : (1 c T ) Vj <0 Vj = l,...,r} 



(63) 



Thus, if we can obtain the extreme rays of the cone T, we have 
the desired representation of the code polytope using (l63l . 

One possible way of doing this is to apply the double 
description method [9|, [10|. A pair of matrices (A, R) is said 
to be a double description (DD) pair if 

Ax t. iff x = Rfi, fi h (64) 

Clearly, (ED and <|62) lead to a DD pair of the cone 7", by 
identifying A = — \P T , x = v, i? = ( vi v 2 . . . v r ) and = 
(/L«i A*2 • ■ • A*r) T - The problem is, given one of the matrices R 
or A, to find the other. In our case, we need to find R given A. 
The double description method is an algorithm which solves 



this problem. Unfortunately, the complexity of the algorithm 
in general grows exponentially with the size of the problem, 
so it will only work reasonably if we keep the code small. 
To apply the DD method, we have used the publicly available 
software implementation cdd+ ifTSl . 

Using cdd+, we were able to obtain a representation of a 
binary constraint node representing the (7, 4) Hamming code 
%t. The matrix R-u 7 has 8 rows and 70 columns, and its 
transpose is given in Figure [5] For example, the first line of 
R^l 7 represents the constraint 



ci 



c-3 + c 5 +c 7 <0 



which is a facet of Q(%7). Note that the constraints < 
d < 1 Vi G {1,2,..., 7} are also implicit in Ru 7 - Using 
this representation together with the lower bound on the frac- 
tional distance in Section I VII we found a randomly-generated 
GLDPC code with 70 variable nodes and 20 constraint nodes, 
each representing a local 1-L-j code, with fractional distance at 
least 11.2418. This code has rate 1/7. The fractional distance 
result guarantees that this code can correct 5 errors using the 
LP decoder. 

Flanagan et al. ifTSI gave an LP formulation suitable 
for nonbinary codes over rings. For a ring 1Z = 
{0, a%, . . . , ai-R.|_i} of size \1Z\, every nonbinary symbol is 
represented using a binary vector of size \1Z\ — 1. If the 
nonbinary symbol is zero in some codeword, the correspond- 
ing binary vector is set to zero. If the nonbinary symbol is 
nonzero, say a 2 , then the i'th element of the binary vector is 
set to 1 and the rest of the binary vector is set to zero. In 
this manner, a nonbinary local codeword Cj is mapped to a 
binary local codeword. The convex hull of all the resulting 
binary codewords corresponding to constraint node j yields 
the fundamental polytope Q(Cj) and the overall polytope is 
the intersection r\j£jQ(Cj), just as in the binary case. The 
difference is that in the nonbinary case, the dimension of the 
polytope is larger, and because of the method of representation, 
Cj, when viewed as a binary code, is not necessarily linear. 

Clearly, using the DD method, it is possible to find (ex- 
perimentally) the representation of any local code, provided 
it is not too large. As an example, using a binary vector of 
length 12, we have found a representation of the fundamen- 
tal polytope of the nonbinary simple parity check code of 
length 4 over GF(4). This polytope has 40 facets, given by 
the rows of the matrix RT„ n which is presented in 

Figure [6] For any particular LDPC code over GF(4) which 
uses SPCg_f(4)(4) as the local code in its constraint nodes, 
we can use the method in Section [VT] to obtain tight lower 
bounds on the fractional distance. The same conclusion applies 
in general when, instead of SPCgj?(4)(4) we take any other 
code, provided that the DD method outputs (in reasonable time 
complexity) a representation of its corresponding fundamental 
polytope. We expect that this will be the case for practical 
constituent codes. 

VIII. Numerical Results 

In the experiments outlined below, Algorithm [TJ was oper- 
ated in the following mode, with respect to the values of the 
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iT 

'SPC GJ , (4) (4) - 
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Fig. 6. The extreme rays of the cone T of the SPC(4) code over GF(4). 



parameters K and eo. Initialize with K = 1000 and eo = 0.01. 
Iterate until e < eo. Then multiply K by 1.26, divide e by 1.26 
and iterate with the new constraints. The process of iterating 
and multiplying and dividing by 1.26 is repeated ten times, so 
that at the end of the process K « 10000 and e ~ 0.001. 

Figure [7] shows simulation results comparing the approxi- 
mate iterative LP decoder with and without merging of check 
nodes, as suggested in Section [IV] In the simulation we picked 
codes at random from Gallager's (3, 6)-regular ensemble of 
length N = 1002. These codes were transmitted over a binary 
symmetric channel. For the merging process, check nodes 
are picked as follows. First, a list of short cycles (up to 
maximum length of 6) in the graph is generated using the 
procedure proposed in fl9l . Based on this list, in each cycle 
we merge the check nodes into one new check node. This 
approach is motivated by Proposition [3] which guarantees an 
unchanged relaxation if the set of merged constraint nodes 
is cycle-free. As a reference, we also plot results for iterative 
belief propagation decoding. It is apparent from the figure that 
despite the improvement due to check node merging, the LP 
decoder exhibits worse performance than the belief propaga- 



tion decoder for values of the channel crossover probability 
greater than 0.045, and better performance for lower values of 
crossover probability. 

In Figure|8] we plot our results from Sections[V]and|Vl] The 
lower bound on the minimum distance derived in Section Mis 
shown in circles. This plot shows, for various block lengths, 
the average value of l m - m (see d49l )) over 10 randomly- 
generated codes, taken from Gallager's (3, 4)-regular ensem- 
ble. We also include improved bounds obtained by merging 
check nodes. Check nodes were chosen for merging as in 
Figure [7] using all cycles of length up to 6. 

Figure [8] also shows an improved lower bound on the 
minimum distance using the bar method outlined at the end of 
Section[V] Clearly, in this case the lower bound is significantly 
improved by allowing pairs of nodes to be examined. Next, 
we plot the lower bound on the fractional distance from 
Section [Vl] The lower bound on the fractional distance is 
calculated with the penalty constant set to B = 10N. In 
this case we set eo in Algorithm [T] to an initial value of 
0.1/-B, instead of 0.01 as previously stated; this more stringent 
setting was used in order to increase the accuracy of our 
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Fig. 7. Simulation results for the BSC comparing the approximate LP decoder with and without merging of check nodes with belief propagation. 
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Fig. 8. Lower bounds on the fractional distance and minimum distance. 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



15 



/ -2 
/ z 


-1 


o 


1 


() 


1 


o 


1 \ 


_2 


o 


-1 


1 


() 


o 


1 


1 


_2 


o 


_1 


1 


1 


1 


o 


o 


_2 


. [ 


o 


1 


1 


o 


1 


o 


_2 


_ 1 


1 


o 


{) 


1 


[ 


o 


_1 


o 


o 


o 


o 


1 


o 


o 


_2 


o 


o 


o 




1 


1 


1 


_i 


o 


o 


o 


o 


o 


1 


o 


_2 


o 


o 


o 


1 


1 


1 




_2 


1 


_[ 


o 


o 


1 


[ 


o 


_2 


o 


I 


-1 


() 


o 


1 


1 


_2 


o 


1 


_1 


1 


1 


o 


o 


_2 


1 


o 


-1 


() 


1 


o 


1 


_2 


1 


o 


-1 


1 


o 


1 


o 


_2 


1 


-1 


() 


1 


o 


() 


1 


-1 


() 


o 


() 


1 


o 


() 


o 


_2 


o 


o 


o 


1 


_1 


1 


1 


_1 


o 


o 


o 


o 


o 


o 


1 


_2 


o 


o 


o 


1 


1 




1 


_2 


. [ 


1 


o 


1 


o 


o 


1 


_2 


o 


1 


1 


o 


o 




1 


_2 


1 


o 


1 


o 




o 


1 


_2 


1 


o 


1 


1 


o 


_1 


o 


_2 


o 


1 


[ 


1 


_[ 





o 


_2 


1 


1 


() 


() 


-1 


1 


o 


_2 


1 


1 


o 


1 


o 


o 




_1 


1 


o 


o 


o 


o 


o 


o 


_2 


1 


1 


o 


-1 


o 


o 


1 


_2 


1 


1 


o 


o 


1 


_1 


o 


-1 


() 


1 


() 


() 


o 


() 


o 


_2 


o 


1 




_1 


1 


o 


o 


_2 


1 


o 


, 


-1 


o 


1 


o 


_2 


1 


o 


1 


() 


1 


() 


-1 


_2 


o 


1 


I 


o 


o 


1 


_1 


-1 


o 


o 


, 


() 


o 


() 


o 





-1 


o 


1 


-1 


o 


-1 


o 


o 


o 


_] 


I 


_1 




o 


o 


o 


o 


_1 


I 


o 


o 


_1 


_1 


o 


_1 


o 


I 


o 




o 


_1 


o 


_1 


1 


o 


_1 


o 


o 


_1 


o 


o 


o 


o 


_] 


1 


_[ 


_1 


o 


o 


o 


o 


. [ 


o 


o 


o 


o 


o 


o 


o 


_1 


_1 


1 


_1 


o 


o 


o 


o 


o 


o 


o 


_1 


o 


1 


_1 


o 


. [ 


o 


o 


_1 


o 


o 


1 


_1 


_1 




o 


o 


o 


o 


1 


-1 


() 


o 


-1 


-1 


o 


1 


o 


_1 


. [ 


o 


_1 


o 


o 


1 


o 


-1 


() 


-1 


() 


-1 


o 


1 


-1 


() 


() 


-1 


-1 


o 


o 


o 


o 


() 


1 


-1 


-1 


-1 


o 


() 


o 


() 


() 


-1 


() 


o 


o 


o 


o 


() 


-1 


-1 


-1 


1 


o 


o 


o 


o 


o 


o 


_1 


o 


o 


_1 


1 


o 


o 




_1 


o 


o 


o 


o 




o 


o 


o 


o 


o 


o 


_1 




1 


_1 


o 


o 


o 


o 


_1 




o 


o 


_1 


1 


o 


_1 


o 




1 


o 


_1 


o 


o 


_1 


o 


. 


o 


_[ 


o 


1 


o 


() 


-1 




() 


o 


1 


-1 


o 


o 


_[ 


, 


_1 


1 


o 


o 





-1 










1 





-1 





-1 







-1 





1 











-1 




















-1 


-1 





1 








-1 





-1 


-1 








-1 


1 








-1 


-1 





-1 








1 





-1 


-1 








1 


-1 





V o 


-1 

















o / 



Fig. 5. The extreme rays of the cone T of the (7,4) Hamming code. 



result in light of the large value of 7 ma JB Finally, exact 
fractional distance results, appearing in asterisks, are included 
in Figure [8] These results were taken from [1] and depict 
exact fractional distance, averaged over a sample of 100 codes. 
With the exception of these exact results from [1], all results 
depict the average over the same 10 randomly-generated codes, 
taken from Gallager's (3, 4)-regular ensemble. Comparing the 
exact results with our lower bound, we see that although 
the randomly-selected codes are not the same, the statistical 
averages indicate that the lower bound is close to the mark for 
B = ION. 

IX. Conclusion 

In this paper we made obtained the following contribu- 
tions to the framework of LP decoding. First, a method for 
improving LP decoding based on merging check nodes was 
proposed. Second, an algorithm for determining a lower bound 
on the minimum distance was presented. This algorithm can 
be improved by the check node merging technique or by the 
greedy procedure introduced in Section [V] This algorithm has 
computation complexity 0(N 2 ), where N is the block length. 
Third, an algorithm for determining a tight lower bound on 
the minimum distance was presented. This algorithm also has 
complexity 0(N 2 ). Fourth, we showed how the fundamental 
polytope can be obtained for GLDPC and nonbinary codes. 
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